Compilation Order Matters: Exploring the Structure of the Space of Compilation Sequences Using Randomized Search Algorithms†

Most modern compilers operate by applying a fixed sequence of code optimizations, called a compilation sequence, to all programs. Compiler writers determine a small set of good, general-purpose, compilation sequences by extensive hand-tuning over particular benchmarks. The compilation sequence makes a significant difference in the quality of the generated code; in particular, we know that a single universal compilation sequence does not produce the best results over all programs [1, 2, 5, 6]. Three questions arise in customizing compilation sequences: (1) What is the incremental benefit of using a customized sequence instead of a universal sequence? (2) What is the average computational cost of constructing a customized sequence? (3) When does the benefit exceed the cost? To answer these questions, we must develop a good understanding of how quality of the generated code varies with the choice of compilation sequence over the entire space of sequences for a given program. In particular, we need to know (1) What percentage of the set of possible compilation sequences falls within a specified neighborhood of the true optimum sequence? (2) How are these nearly optimal sequences distributed in the sequence space? Do good sequences cluster in particular regions of the space? Or are they distributed evenly? (3) Is the sequence space riddled with shallow local minima? Can random sampling in the space reliably achieve near-optimal solutions? More importantly, we need to know if there are structural properties shared by compilation sequence spaces for a broad range of pro-